1. Field of the Invention
The present invention relates to an image taking and/or displaying system which can take and/or display a stereoscopic image or a wide image. More particularly, the invention relates to an image taking and/or displaying system provided with a multi-eye camera apparatus having a plurality of camera units and a displaying device which can permit an observer to observe an enlarged screen of virtual images of taken images through a compact display unit mounted on the observer""s head.
2. Related Background Art
Recently used to obtain camera images close to the human vision are stereoscopic image taking apparatus which can obtain camera images by taking images of an object in a same direction, for example, through two camera units. As described in Japanese Laid-open Patent Application No. 62-122493, a conventional stereoscopic image taking apparatus employs a stereoscopic image taking and focusing method for tracking an intersection between lens optical axes of two video cameras (hereinafter referred to as cameras) and performing focusing, based on a base length and an angle between the two cameras.
There are also various proposals of stereoscopic image displaying apparatus for displaying images taken by such multi-eye cameras. Among these stereoscopic image displaying apparatus is a so-called head-mounted display (hereinafter referred to as HMD), which is a display apparatus which can permit the observer to observe an enlarged screen of virtual images of taken images through a compact display unit mounted on the observer""s head.
As for the image taking and/or displaying system mainly composed of the multi-eye camera and the HMD, there are proposed image taking and/or displaying systems, which are different from the prior art systems simply for taking and appreciating scenic images. Such systems are disclosed, for example, in U.S. Pat. No. 2,711,594, U.S. Pat. No. 3,205,303, and U.S. Pat. No. 3,670,097. In the systems, a sensor for detecting a moving direction of the observer""s face is provided in the HMD, information about the moving direction is sent from the sensor to a control unit for controlling the camera posture, and a taking field of camera is changed so as to track the moving direction of the observer""s face. According to this arrangement, when the observer simply changes the face into a desired direction, the camera is directed in the direction and sends an image of the external field in that direction as a camera signal to the HMD on the observer whereby the observer can observe the image in that direction.
As one close to the above system there is studied a tele-existence system in which a multi-eye camera is set on a robot equipped with a manipulator and set at a remote place or at a dangerous place, and then a stereoscopic image signal from the camera is received to be displayed on the HMD on the observer, thus enabling operations at the remote place.
The present applicant or assignee has proposed a multi-eye camera which can take a three-dimensional image and a horizontally wide image in Japanese Patent Application No. 5-211777 and an HMD display apparatus which can display a three-dimensional image and a horizontally wide image in Japanese Patent Application No. 5-92113.
Although the above conventional examples showed the system arrangement of the multi-eye camera and the HMD display apparatus, proposing the control of the camera posture and the remote operations as monitoring the stereoscopic image, they failed to suggest new image taking and/or displaying technology as a three-dimensional image taking and/or displaying system.
The conventional stereoscopic image taking and/or displaying systems had the following problems.
(1) The display apparatus cannot reproduce a change of taking conditions in image-taking, such as a change of a convergence angle. Then displayed images cannot reflect photographer""s purpose, so that the observer has to watch unnatural stereoscopic images, which would result in physiological fatigue after long-time appreciation.
(2) A stereoscopic image taken through the multi-eye camera in image-taking cannot be correctly displayed on the spot as a stereoscopic image according to photographic conditions and photographic purpose. Therefore, the photographer cannot check by himself whether the image correctly reflects the photographic purpose.
As described above, the present applicant or assignee has proposed the multi-eye image taking camera which can take three-dimensional images and two-dimensional wide images and the HMD display apparatus which can display three-dimensional images and two-dimensional, horizontally wide images, but there has been no system proposed yet taking advantage of the features of the image taking camera and HMD display apparatus.
An object of the present invention is to provide an image taking and/or displaying system which can display an image correctly reflecting photographer""s purpose.
An image taking and/or displaying system of the present invention is an image taking and/or displaying system which has an image-taking camera apparatus having image-taking means for producing a plurality of image signals corresponding to optical images from different visual points, and a display apparatus for guiding a pair of images displayed on a pair of image displaying means through a pair of projection optical systems to the observer""s eyes so that the observer can observe enlarged, virtual images of said pair of images by the respective eyes, and in which the plurality of image signals from said image-taking camera apparatus are displayed on the pair of image displaying means in said displaying apparatus to permit the observer to observe them; or an image taking and/or displaying system which has an image-taking camera apparatus having image-taking means for producing a plurality of image signals corresponding to optical images from different visual points, and a displaying apparatus for guiding a pair of images displayed on a pair of image displaying means through a pair of projection optical systems to the observer""s eyes so that the observer can observe enlarged, virtual images of said pair of images by the respective eyes and controlling positions of said displaying means relative to said projection optical systems or a focusing state of said projection optical systems whereby said displaying apparatus can control spatial positions of planes of said two virtual images between a superimposed state where the two images are superimposed on each other and an adjacent state where the two images are juxtaposed adjacent to each other, and in which the plurality of image signals from said image-taking camera apparatus are displayed on the pair of displaying means in said displaying apparatus to permit the observer to observe them. Further, the system is characterized in that control information of said image-taking camera apparatus is supplied to said displaying apparatus in addition to the plurality of image signals from said image-taking camera apparatus, and then the relative, spatial positions of said pair of enlarged, virtual images in said displaying apparatus are changed in accordance with said control information.
Further, an image taking and/or displaying system of the present invention is an image taking and/or displaying system which has an image-taking camera apparatus having image-taking means for producing a plurality of image signals corresponding to optical images from different visual points, an image information recording apparatus for recording the plurality of image signals from said image-taking camera apparatus in an image information recording medium, and a displaying apparatus for guiding a pair of images displayed on a pair of image displaying means through a pair of projection optical systems to the observer""s eyes so that the observer can observe enlarged, virtual images of said pair of images by the respective eyes, and in which the plurality of image signals reproduced from the image information recording medium in said image information recording apparatus are displayed on the pair of displaying means in said displaying apparatus to permit the observer to observe them; or an image taking and/or displaying system which has an image-taking camera apparatus having image-taking means for producing a plurality of image signals corresponding to optical images from different visual points, an image information recording apparatus for recording the plurality of image signals from said image-taking camera apparatus in an image information recording medium, and a displaying apparatus for guiding a pair of images displayed on a pair of image displaying means through a pair of projection optical systems to the observer""s eyes so that the observer can observe enlarged, virtual images of said pair of images by the respective eyes and controlling positions of said displaying means relative to said projection optical systems or a focusing state of said projection optical systems whereby said displaying apparatus can control spatial positions of planes of said two virtual images between a superimposed state where the two images are superimposed on each other and an adjacent state where the two images are juxtaposed adjacent to each other, and in which the plurality of image signals reproduced from the image information recording medium in said image information recording apparatus are displayed on the pair of displaying means in said displaying apparatus to permit the observer to observe them. Further, this system is characterized in that control information of said image-taking camera apparatus is also recorded in the image information recording medium in said image information recording apparatus in addition to said plurality of image signals and that the plurality of image signals and said control information reproduced from said image information recording medium are supplied to said displaying apparatus, and then the relative, spatial positions of said pair of enlarged, virtual images in said displaying apparatus are changed in accordance with said control information.
The present invention is intended to solve the above problems by proposing a system taking full advantage of the features of the multi-eye image-taking camera which can take two-dimensional wide images as well as three-dimensional images and the HMD display apparatus which can display not only three-dimensional images but also two-dimensional, horizontal wide images.
Namely, the present invention is intended to solve the above problems (1) and (2) in such a manner that the control information of the multi-eye camera in taking images is sent to the HMD display apparatus upon transmission of the image signals and, based on the control information, the positions of the display units in the HMD displaying apparatus are controlled or the focusing state of the projection optical systems in the HMD displaying apparatus are controlled so as to display the three-dimensional image correctly reflecting the photographer""s purpose.
Also, the present invention involves such an arrangement that upon transmission of the image signals control information indicating whether the multi-eye camera in taking images is in a state where it is taking a two-dimensional (2D) image or in a state where it is taking a stereoscopic or three-dimensional (3D) image is sent to the HMD display apparatus, the control is performed based on the control information so as to control the positions of the display units in the HMD display apparatus or the focusing state of the projection optical systems in the HMD display apparatus, and display is changed over in the HMD between display of 2D wide image and display of 3D image to enable image display correctly reflecting the photographer""s purpose, whereby the observer can enjoy the new image apparatus which can display the 3D image, the 2D wide image, or a mixture thereof.
Incidentally, the aforementioned conventional examples are so arranged that the convergence angle of the two cameras is adjusted according to an object in taking images thereof by the image-taking apparatus, but the display apparatus is so arranged that the display units cannot be adjusted to change a convergence angle of the two display units, based on the state of the cameras in taking images, which sometimes forces the observer to observe the images with convergence completely different from that in taking images. For example, even with an image of an object distant at about 1 m, the observer could observe the image with such convergence as if to observe an infinite view. Conversely, even though the image is of an infinite view, the observer could observe the image with such convergence as if to observe an object at 1 m in front thereof. Thus, the conventional systems had such a disadvantage that the stereoscopic effect was emphasized more than necessary or that the image was unnatural, which readily made the eyes very tired. Attempts have been made to fix the convergence approximately matching with the images and to allow manual adjustment of convergence. However, the same disadvantage still remains, because the convergence is not always adjusted to match with the taking conditions.
It is, therefore, another object of the present invention to provide an image taking and/or displaying system which can obtain natural images and which can relieve the stress on the eyes.
To achieve this object, the present invention involves such an arrangement that a state of cameras in taking images is detected and a state of display units is adjusted based on the detected information. Specifically, the image taking and/or displaying system is arranged based on the following (1) to (5).
(1) The system is an image taking and/or displaying system in which images taken by an image-taking apparatus having a plurality of video cameras are displayed by a displaying apparatus having a same or less number of display portions as or than the number of said video cameras, in which said image-taking apparatus is provided with camera state detecting means for detecting a state of said video cameras in taking images and said displaying apparatus is provided with controlling means for controlling a state of said display portions in accordance with an output from said camera state detecting means.
(2) The system is provided with adjusting means for advancing camera state information detected by the camera state detecting means relative to image information.
(3) The camera state information is information about the convergence angle of the video cameras.
(4) The adjusting means is for advancing a phase of the camera state information relative to the image information by a timing to read the information from a memory.
(5) The adjusting means is for advancing a phase of the camera state information relative to the image information by arranged positions of reproducing heads for reading the information from a recording medium.
According to either one of the above (1) to (5), the state of the display portions is controlled according to the state of the video cameras. According to either one of the above (2) to (5), there remains no time lag of change in the state of the display portions with respect to the images.
Another object of the present invention is to provide a multi-eye image-taking apparatus which can make a display apparatus display natural images whereby the stress on the eyes can be relieved.
To achieve the above object, the present invention involves such an arrangement that the multi-eye image-taking apparatus outputs camera state information with a phase advancing with respect to the image information together therewith.
Specifically, the multi-eye image-taking apparatus is arranged based on the following (1) and (2).
(1) The multi-eye image-taking apparatus is arranged to be provided with a plurality of video cameras, convergence angle adjusting means for adjusting a convergence angle of the video cameras, convergence angle detecting means for detecting the convergence angle adjusted by the convergence angle adjusting means, and adjusting means for advancing information on the convergence angle detected by the convergence angle detecting means ahead of the image information from the video cameras.
(2) The multi-eye image-taking apparatus is arranged to be further provided with mode changeover means for changeover between a stereo mode and a panorama mode, whereby an initial value of the convergence angle adjusting means is set in accordance with the mode changeover by the mode changeover means.
According to either one of the above (1) and (2), the multi-eye image-taking apparatus can obtain the image information and the convergence angle information going ahead of the image information. According to the above (2), the mode can be changed over between the stereo mode and the panorama mode, so that the initial value of convergence angle can be set in accordance with the mode changeover.